Field of the Invention
The present invention relates to an image processing apparatus, a control method, and a recording medium, and particularly to a technique for generating an image focused on an arbitrary subject distance from image data after capturing an image.
Description of the Related Art
Recently, a technique has been proposed for recording an intensity distribution and a traveling direction of light as output data when an image is captured with an imaging apparatus, such as a digital camera, in order to generate an image in which an arbitrary subject distance is focused from the output data after recording.
In Ren. Ng, et al., “Light Field Photography with a Hand-Held Plenoptic Camera”, Stanford Tech Report CTSR 2005-02, a method is disclosed in which light incident from various directions is separated and recorded by forming light flux, that has passed through different divided pupil regions of an imaging lens, into an image on pixels (photoelectric conversion elements) of an imaging element via a microlens array. In the output data obtained in this manner (Light Field Data; hereinafter, LF data), adjacent pixels record light flux incident from different directions.
By extracting light flux of the same direction from pixels associated with each microlens, an image captured in that direction can be generated from the LF data. Also, pixels of an image in which a specific subject distance is focused can be generated (reconstructed) after capturing an image, by setting an arbitrary subject distance and adding the output of pixels that record light flux that has passed through one point on a focal plane corresponding to that subject distance.
With typical image viewing performed using a display apparatus, a user can enlarge an arbitrary region and check the captured state of the subject in the region. In an image captured by a regular imaging apparatus, the focused state of the subject, or in other words, the sharpness of the subject, is determined by the setting conditions and the like of the optical system at the time of capturing an image, and thus whether or not a user can recognize the details of the subject in a region of an image by enlarging the region is dependent on the focused state (of the subject).
Meanwhile, because a reconstruction image in which an arbitrary subject distance is focused can be generated from LF data obtained by capturing an image, for example, in the case where a specific region of the reconstruction image is enlarged, it is possible to enlarge and present the reconstruction image in which a subject in the region is focused.
However, the desired effect of an operation for checking the captured state of a subject by enlargement may differ depending on the user. For example, in the case where a user wants to output from LF data a reconstruction image of a state in which subjects other than the main subject are blurred, such as a portrait, enlarged display may be used as a way of checking that the subjects are in a desired blurred state. Also, for example, in the case where a user wants to output from LF data a reconstruction image in which a specific subject is not included in the depth of field, enlarged display may be used as a way of checking that the subject is not visually recognizable. Thus, generating an image in which a subject is focused, in a region where enlarged display is performed, does not necessarily coincide with the user's intention for performing the operation.
Also, although it is preferable, in the case where a user wants to check the details of a subject in a predetermined region, for example, that an image in which the subject is focused is presented at the time of enlarged display, the reconstruction image that a user desires to be presented when an operation for restoring the original magnification is performed may differ depending on the user. For example, in the case where a user only intended to check the details of a subject, it is preferable that a reconstruction image in the focused state before enlargement is presented after a reduction operation for restoring the original magnification is performed. On the other hand, for example, in the case where a user finds a subject that he or she wants to be focused by enlarged display and the focused state is changed so that the subject is focused, it is preferable that after the reduction operation is performed, a reconstruction image in a focused state similar to the state at the time of enlarged display is presented.